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ABSTRACT 

Optical  sensors  aboard  space  vehicles  designated  to  perform  seeker  functions  need  to  generate  multispectral  images  in  the 
mid-wave  infrared  (MWIR)  and  long-wave  infrared  (LWIR)  spectral  regions  in  order  to  investigate  and  classify  man-made 
space  objects,  and  to  distinguish  them  relative  to  the  interfering  scene  clutter.  The  spectral  imager  part  of  the  sensor  collects 
spectral  signatures  of  the  observed  objects  in  order  to  extract  information  on  surface  emissivity  and  target  temperature,  both 
important  parameters  for  object-discrimination  algorithms.  The  Adaptive  Spectral  Imager  described  in  this  paper  fulfills 
two  functions  simultaneously:  one  output  produces  instantaneous  two-dimensional  polychromatic  imagery  for  object 
acquisition  and  tracking,  while  the  other  output  produces  multispectral  images  for  object  discrimination  and  classification. 
The  spectral  and  temporal  resolution  of  the  data  produced  by  the  spectral  imager  are  adjustable  in  real  time,  making  it 
possible  to  achieve  optimum  tradeoff  between  different  sensing  functions  to  match  dynamic  monitoring  requirements 
during  a  mission.  The  system  has  high  optical  collection  efficiency,  with  output  data  rates  limited  only  by  the  readout  speed 
of  the  detector  array.  The  instrument  has  no  macro-scale  moving  parts,  and  can  be  built  in  a  robust,  small-volume  and 
lightweight  package,  suitable  for  integration  with  space  vehicles.  The  technology  is  also  applicable  to  multispectral 
imaging  applications  in  diverse  areas  such  as  surveillance,  agriculture,  process  control,  and  biomedical  imaging,  and  can  be 
adapted  for  use  in  any  spectral  domain  from  the  ultraviolet  (UV)  to  the  LWIR  region. 

Keywords:  Multispectral,  hyperspectral,  imaging,  infrared,  space  vehicle 

1.  INTRODUCTION 

Future  optical  sensors  aboard  space  vehicles  will,  in  addition  to  other  functions,  need  to  investigate  and  classify  other  man¬ 
made  objects  within  their  field  of  view,  and  to  distinguish  them  against  the  interfering  scene  clutter  (e.g.  contribution  from 
Earth  radiation  and  light  reflection).  This  need  can  be  fulfilled  by  generating  and  analyzing  multi  spectral/hyper  spectral 
images  of  those  objects.  The  Adaptive  Spectral  Imager  described  in  this  paper  produces  instantaneous  two-dimensional 
polychromatic  (spectrally  unresolved)  imagery  for  object  acquisition  and  tracking,  and,  simultaneously,  multispectral 
images  for  object  discrimination  and  classification.  By  using  the  mid- wave  infrared  (MWIR)  and  long- wave  infrared 
(LWIR)  spectral  signatures  of  the  object,  it  is  possible  to  extract  the  information  on  object  temperature  and  surface 
material,  for  objects  showing  significant  temperature  contrast  relative  to  the  surrounding  scene. 

The  spectral  and  temporal  resolution  of  the  data  produced  by  the  spectral  imager  are  adjustable  in  real  time,  making  it 
possible  to  achieve  the  optimum  tradeoff  between  different  sensing  functions  in  order  to  meet  the  dynamic  monitoring 
requirements  of  this  application.  The  system  needs  to  produce  near-diffraction  limited  imagery  with  as  high  optical 
collection  efficiency  as  possible,  and  with  a  provision  for  spectral  multiplexing  (collecting  data  on  many  wavelengths 
simultaneously).  The  instrument  should  be  designed  as  a  robust,  small- volume,  lightweight  and  low-power  consumption 
device  with  no  macro-scale  moving  parts  in  order  to  make  it  suitable  for  integration  into  space  vehicles. 

In  this  paper  we  describe  the  design  of  the  Adaptive  Imaging  Spectrometer,  the  results  of  performance  modeling,  the  unique 
optical  system  of  the  instrument  and  the  results  of  the  experimental  work  with  a  feasibility  demonstration  prototype. 


:pajo@spectral.com;  phone:  781  273  4770  ;  fax:  781  270  1161  ;  www.spectral.com 


2.  INSTRUMENT  CONCEPT 


The  principles  of  operation  of  the  Adaptive  Imaging  Spectrometer  are  illustrated  on  Fig.  1.  The  instrument  uses  two 
polychromators  (or  a  single  polychromator  in  two  passes)  and  a  spatial  light  modulator  (SLM)  placed  between  them. 
There  is  no  physical  slit  in  the  system;  the  system  operates  as  a  staring  two-dimensional  (2D)  imaging  device.  The  first 
polychromator  disperses  and  images  all  spectral  components  onto  the  SLM.  The  SLM  spectrally  encodes  the  image  by 
modulating  the  intensity  of  selected  spectral  bands  in  the  dispersed  image.  This  operation  is  equivalent  to  applying  a 
mathematical  transform  to  the  spectrally  resolved  image,  and  is  easy  to  do  with  a  digitally  controlled  SLM.  The  encoded 
image  is  spectrally  recombined  by  the  second  polychromator  and  imaged  onto  a  2D  focal  plane  array  (FPA) 
photodetector  (Fig.  1). 

Spectral  encoding  is  achieved  by  transmission,  rejection  or  intensity  modulation  of  particular  wavelengths.  A  2D  array 
SLM  whose  pixels  are  individually  addressable  (and  small  relative  to  the  resolution  element),  can  be  programmed  to 
generate  a  dynamic  series  of  spatial  patterns  (masks)  that  implement  a  variety  of  transform  functions.  Transform 
functions  include  traditional  single-slit  wavelength  scanning,  simple  multiple-slit  filtering,  multiplexed  Hadamard 
transform  spectroscopy,  and  “matched  filter”  transform,  where  the  SLM  is  used  to  produce  a  spectral  mask  that  matches 
known  distinctive  spectral  features  of  the  target.  An  approach  even  more  advanced  is  to  construct  a  “filter”  mask  over 
the  SLM  that  mimics  the  multivariate  regression  vector  for  a  specific  principal  component.  In  this  way,  the  optical 
processing  by  the  SLM  becomes  equivalent  to  numerically  projecting  the  input  spectrum  onto  the  principal  component 
chosen  by  the  SLM  mask.  The  detector  signal  then  becomes  the  “score”  of  the  measured  spectrum  relative  to  a  selected 
principal  component.  A  number  of  principal  components  can  be  investigated  in  quick  succession,  by  cycling  their 
respective  multivariate  regression  vector  masks  over  the  SLM. 
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Figure  1:  Principles  of  operation. 


The  Adaptive  Spectral  Imager  described  here  uses  a  reflective  Digital  Micromirror  Array  (DMA)  as  the  modulator  that 
performs  spectral  encoding  of  two-dimensional  images.  For  the  case  where  remote  measurement  of  temperature  and 
emissivity  of  the  object  is  the  main  goal,  the  spectrometer  operates  in  the  MWIR/LWIR  range,  and  typically  acquires 
broadband  spectra  in  a  “survey”  (not  filtered)  mode  (Fig.  1).  Such  mode  of  operation  is  best  implemented  by  the 
Hadamard  transform  due  to  its  multiplexing  capability  and  its  processing  speed1.  For  Hadamard  transform,  the  DMA  is 
programmed  with  a  series  of  binary  masks  based  on  Simplex  (S)  matrices,  where  elements  0  and  1  translate  into  on-off 
modulation  of  individual  micromirror  pixels  of  the  DMA1.  Recording  a  A-channel  spectrum  in  this  way  requires  N 
masks  based  on  a  set  of  N  matrices  resulting  in  N1/2  /2  multiplex  (Felgett’s)  advantage  in  the  signal  to  noise  ratio  (SNR) 
relative  to  the  slit-scanning  approach.  The  multiplexing  approach  collects  the  light  contained  in  approximately  half  of 
all  the  wavelengths  simultaneously  at  any  point  in  time,  thereby  providing  the  much  needed  photon  collection  efficiency 
in  the  MWIR/LWIR.  The  recorded  images  are  decoded  by  the  inverse  Hadamard  transform  to  provide  the  spectrum  for 
each  individual  pixel  (Fig.  1). 


3.  INSTRUMENT  DESIGN 


The  Adaptive  Spectral  Imager  for  space  vehicles  relies  on  dedicated  foreoptics  (typically  a  telescope)  to  deliver  the 
image  into  its  input  field  (Fig.  2).  It  also  uses  one  or  more  2D  detector  arrays  (FPA)  to  convert  the  photons  into 
electrical  signals  that  can  be  further  processed  by  the  sensor’s  electronics  and  analyzed  by  onboard  computers  (Fig.  2). 

The  imager  provides  data  for  two  functions  simultaneously,  based  on  the  way  the  output  from  the  FPA  is  processed:  one 
output  produces  standard  unresolved  polychromatic  images  at  the  maximum  frame  rate  of  the  FPA,  which  the  space 
vehicle  can  use  for  object  acquisition  and  tracking.  The  other  output,  processed  simultaneously  with  the  first  one 
produces  multispectral  images  for  object  discrimination  and  classification.  By  applying  selected  functions  with  a 
selected  set  of  parameters  to  the  spectral  modulator,  the  type  of  information  and  spectral  and  temporal  resolution  of  the 
data  produced  by  the  spectrometer  can  be  adjusted  in  real  time  (Figs.  1  and  2). 


Figure  2:  System  design. 


The  conceptual  block  diagram  of  the  software  package  that  controls  the  instrument,  collects  data,  and  delivers  spatial  and 
spectral  information  to  higher-level  external  programs  is  presented  on  Fig.  2.  The  data  stream  captured  by  the  FPA  is 
routed  to  parallel  spatial  and  spectral  processing.  The  spatial  image  is  based  on  the  integrated  intensity  of  all  open 
spectral  channels  (about  half  of  the  total  for  the  Hadamard  transform).  In  order  to  make  this  image  equivalent  as  much 
as  possible  to  the  panchromatic  image  that  would  be  captured  by  a  staring  FPA,  additional  simple  processing  can  be 
applied  in  the  form  of  interpolation  and  fitting  pixel  intensities.  The  output  of  this  procedure  is  high  quality  two- 
dimensional  imagery  used  in  real  time  by  the  target-tracking  subsystem  of  the  vehicle. 

In  the  case  of  collecting  broad  spectra  with  the  Hadamard  transform-based  modulation,  the  spectral  processing  software 
starts  with  pixel-by-pixel  processing  of  the  data.  The  reduction  of  Hadamard  encodegrams  into  spectra  is  accomplished 
by  applying  the  inverse  Hadamard  transform1.  While  this  operation  in  a  general  case  could  be  rather  time-consuming, 
due  to  the  large  number  of  matrix  inversions  involved,  the  inversion  of  binary  square  cyclic  S-matrices  is  extremely 
efficient  and  fast  (orders  of  magnitude  faster  than  e.g.  Fast  Fourier  Transform  (FFT)  calculations).  The  resulting 
datacubes  are  arranged  in  a  band- sequential  format,  standard  for  processing  multispectral  and  hyperspectral  imagery.  In 
the  case  of  temperature/emissivity  measurements,  based  on  multichannel  infrared  spectrothermometry,  the  Planck 
function  is  fitted  to  the  data  with  great  accuracy,  due  to  a  large  number  of  channels  (10-100)  in  a  wide  spectral  range 
(entire  MWIR  or  LWIR  region,  or  both).  Such  rich  spectral  information  makes  it  possible  to  successfully  unmix  the 
temperature  and  the  emissivity  of  the  object,  thereby  providing  data  on  the  nature  of  the  radiating  surface.  Additionally, 
the  abundance  of  spectral  data  also  helps  separate  the  target  signature  from  other,  often  complex  and  confounding  optical 
phenomena  seen  by  the  sensor,  which  can  easily  contaminate  the  target’s  spectral  signature. 

The  Adaptive  Imaging  Spectrometer  must  be  able  to  detect  small  objects  at  great  distances,  meaning  that  the  object 
image  might  be  smaller  than  a  single  pixel  on  the  FPA  and  very  faint.  Therefore,  the  optical  system  needs  to  be  both 
near-diffraction  limited  and  have  a  large  collection  efficiency.  As  described  above  related  to  Fig.  1,  two  high  quality 
polychromators  are  needed,  in  a  back-to-back  configuration,  making  the  optical  design  not  a  simple  task,  especially 
under  the  size  and  weight  constraints  of  space  based  instrument.  Candidate  spectrometer  layouts  for  such  a  system  can 
be  based  either  on  a  pair  of  Offner  spectrometers  with  discrete  elements2’3,  a  pair  of  curved-prism  spectrometers4,  or  a 
design  based  on  an  integrated  double  Offner  relay,  where  elements  are  shared  between  the  dispersion  and  the 
recombination  paths.  We  have  analyzed  the  first  two  options  and  found  them  mechanically  complex  and  bulky  for  a 
space-based  instrument.  Therefore,  we  have  embarked  on  a  task  to  design  a  dedicated  optimized  optical  system  for 
adaptive  spectrometers,  based  on  the  third  option  (Fig.  3). 
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Figure  3:  Double-pass  adaptive  imaging  spectrometer:  solid  model. 


The  optical  system  of  Fig.  3  uses  a  novel  and  simple  dual-pass  spectrograph  design,  defined  by  two  curved  elements,  a 
grating  and  a  single  spherical  reflector.  The  spectrograph  uses  a  near-concentric  configuration,  similar  to  the  Offner 
relay2.  Like  all  Offner  spectrographs,  it  uses  a  curved  grating  with  a  radius  of  curvature  of  one-half  that  of  the  spherical 
mirror  to  produce  a  dispersed  image  with  1:1  magnification.  The  input  and  output  images  are  located  in  a  plane 
containing  the  center  of  curvature,  but  displaced  from  the  center  of  curvature.  The  design  differs  from  the  traditional 
design  in  that  it  uses  a  vertical  rather  than  lateral  orientation  of  the  grating,  meaning  that  the  light  is  dispersed  in  a 
direction  perpendicular  to  the  image  displacement.  This  places  the  dispersed  image  near  the  center  of  curvature  in  the 
dispersion  plane,  thereby  minimizing  the  image  aberrations.  By  contrast,  other  designs  using  the  lateral  orientation3, 
disperse  the  light  along  the  displacement  direction.  This  results  in  very  good  imaging  in  the  slit  dimension,  but  large 
image  aberrations  in  the  direction  perpendicular  to  the  slit.  While  a  dual-pass  spectrograph  can  be  made  by  combining 
two  lateral  Offner  spectrographs5,6,  a  much  higher  level  of  complexity  is  required.  We  have  found  that  we  can  achieve 
near-diffraction  limited  image  quality  with  just  two  curved  surfaces  using  the  vertical  configuration.  By  contrast,  six 
curved  surfaces  (two  gratings  and  three  spherical  reflectors)  are  required  to  achieve  the  same  image  quality  with  the 
lateral  configuration. 

We  are  currently  assembling  a  breadboard  prototype  of  the  dual-pass  spectrograph  for  testing.  Figs.  3  and  4  show  a 
solid  model  and  a  plane  view  of  the  optical  system,  respectively.  The  spectrograph  is  defined  by  the  grating  and  the 
spherical  mirror.  The  input  and  output  of  the  spectrograph  are  displaced  from  the  center  of  curvature  along  the  X-axis, 
and  the  light  is  dispersed  along  the  Y  Axis.  The  light  makes  two  complete  passes  through  the  relay,  once  to  disperse  the 
light,  once  to  recombine  the  light.  Light  entering  the  spectrograph  at  the  object  plane  passes  through  the  spectrograph  to 
form  a  dispersed  image  on  the  DMA.  The  DMA  reflects  the  input  light  at  a  24-degree  angle  in  the  YZ  plane  and  passes  it 
back  through  the  spectrograph,  which  recombines  the  various  wavelengths.  A  turning  mirror  picks  off  the  light  returning 
light  and  forms  a  recombined  image  on  the  detector.  Without  the  turning  mirror,  the  recombined  image  would  overlap 
the  input  image  in  the  object  plane. 

For  convenience  in  testing,  we  are  using  relatively  large  optics  and  a  standard  camera  dewar.  The  concave  mirror  has  a 
diameter  of  180mm  and  a  radius  of  curvature  of  200mm.  The  grating  is  a  19mm  x  45mm  convex  rectangle,  with  the 
radius  of  curvature  of  100  mm.  The  effective  F-number  of  the  system  is  F/2.8.  The  modulator  is  a  rectangle  of 
14x10. 5mm,  rotated  45  degrees  about  the  Z-axis.  The  wavelength  range  is  7-1 3pm  in  the  first  order,  3.5pm-  5pm 
(6.5pm)  in  the  second.  The  FPA  is  a  liquid  nitrogen  cooled  array  of  256x256  square  pixels  of  30pm  x  30pm  each  (8mm 
x  8mm  total  size).  The  recombined  image  is  nearly-diffraction  limited.  The  geometric  spot  size  radii  are  between  10 
and  20  pm  rms,  over  the  entire  8x8  mm  field  of  view. 


(a)  (b) 


Figure  4:  Double-pass  adaptive  imaging  spectrometer  optics:  plane  view  in  the  dispersion  plane  YZ  (a)  and  in  the  offset  plane  XZ  (b). 


The  spatial  light  modulator  used  in  our  system  is  a  modified  DMA  chip  by  Texas  Instruments  (DMD  0.7  XGA  12° 
DDR).  The  array  consists  of  1024  x  768  aluminum  coated  individual  micromirrors  of  14pm  x  14pm  each,  capable  of 
±12°  tilt  relative  to  the  diagonal.  Mirrors  are  individually  addressable  through  a  driver  circuit  available  from  the 
manufacturer.  Modulation  bandwidth  close  to  10kHz  is  possible  with  a  high  speed  version  of  the  driver  circuitry.  For  IR 
applications,  the  original  glass  window  needs  to  be  replaced  with  a  ZnSe  IR- transparent  window. 

The  size  of  individual  micromirrors  for  the  TI  DMA  chip  has  been  chosen  to  produce  negligible  diffraction  effects  in  the 
visible  range,  where  the  chip  is  typically  used.  However,  at  long  7pm  -13pm  wavelengths  used  here,  the  diffraction 
effects  can  seriously  degrade  the  optical  performance,  if  an  optical  system  with  a  small  collection  angle  is  used.  We 
have  modeled  the  diffraction  effects  within  a  system  of  the  kind  presented  on  Fig.  3  in  the  range  3  pm- 13  pm  and  found  a 
strong  dependence  of  the  image  quality  on  the  collection  angle  of  the  optics.  However,  the  model  shows  the  collection 
angle  afforded  by  F/2.8  of  the  system  of  Fig.  3  is  large  enough  to  produce  the  intensity  ripples  in  the  recombined  focal 
plane  not  larger  than  several  percent  of  the  maximum  intensity. 

Assuming  that  the  image  generated  on  the  FPA  fills  a  detector  pixel  (this  corresponds  to  the  spatial  resolution  of  0.5m  at 
the  distance  of  100km),  with  F/2.8  optics,  the  spectral  resolution  of  10  channels,  and  a  typical  LWIR  FPA  sensitivity 
radiometric  calculations  for  our  LWIR  adaptive  spectral  imager  suggest  the  signal-to-noise  ratio  (SNR)  around  28dB  per 
collected  spectrum,  which  is  high  enough  to  accomplish  object  characterization. 

4.  EXPERIMENTAL  MODEL 

A  feasibility  prototype  of  the  Adaptive  Spectral  Imager  as  applied  to  standoff  imaging  thermography  has  been 
demonstrated  so  far  in  the  VIS/NIR  spectral  region  600nm  -  llOOnm  where  a  silicon  CCD  camera  was  used.  Such 
system  is  aimed  at  imaging  thermography  of  objects  at  temperatures  above  1000K.  An  optical  system  based  on  refractive 
and  transmissive  components  was  assembled  for  the  purpose.  Data  acquisition  and  processing  software  was  developed 
for  the  control  of  the  DMA,  capturing  data  from  the  CCD  camera,  and  for  data  reduction.  Spectral  calibration  with 
known  input  wavelength(s)  produced  the  instrument  calibration  factor  of  0.4  lnm  per  micromirror  of  the  DMA  array. 

The  imaging  thermography  capability  of  the  prototype  was  investigated  by  using  three  blackbody  objects  at  different 
temperatures  placed  into  the  instrument’s  field  of  view  (Fig.  5).  The  blackbodies  were  represented  by  electrically  heated 
resistors  kept  at  constant  temperature.  One  of  the  sources  used  (Object  1,  Fig.  5)  was  a  calibrated  1250K  blackbody 
source.  A  seven-element  Hadamard  mask  was  used  to  produce  seven-band  spectra  of  each  object  with  adequate  spectral 
resolution  for  temperature  measurement  at  a  fast  update  rate.  Fig.  5  shows  a  dynamic  sequence  of  data  collected  with 
our  adaptive  spectral  imager.  The  left-hand  side  shows  the  time  sequence  of  spatial  (spectrally  combined)  images,  as 
generated  after  each  Hadamard  transform  mask  was  applied  to  the  DMA.  Since  half  of  all  the  spectral  channels  are  open 
at  any  moment  in  time,  and  since  the  FPA  readout  is  updated  after  every  mask,  the  spectrally  recombined  image  on  the 
FPA  shows  a  near-panchromatic  view  of  the  objects.  The  right-hand  side  of  Fig.  5  shows  the  raw  (instrument  function 
included)  Hadamard-decoded  spectra  of  three  objects,  updated  partially  after  each  mask,  and  updated  fully  after  each 
complete  cycle  of  7  masks.  Data  are  scaled  by  an  approximate  instrument  function,  to  compensate  for  the  detector 
falloff  at  wavelengths  greater  than  900  nm.  Figure  6  shows  the  same  data  along  with  a  fit  to  the  Planck  function  that 
reveals  the  approximate  temperature  of  the  object.  Note  that  the  vertical  (signal  intensity)  scale  does  not  represent  the 
radiance  of  the  object,  since  the  solid  angle  has  not  been  calibrated.  Based  on  this  approach  we  found  the  approximate 
temperatures  of  the  three  objects  to  be  1400K,  1600K  and  1900K  (Fig.  6). 

After  experimentally  demonstrating  the  ability  to  perform  imaging  thermography  of  high-temperature  objects  in  the 
VIS/NIR  spectral  region  with  the  simple  demonstration  experiment  described,  we  have  focused  at  building  a  much  more 
technologically  complex  MWIR/LWIR  system  described  above.  The  system  will  be  capable  of  imaging  thermography 
at  temperatures  around  and  above  300K.  The  system,  which  is  based  on  the  design  described  in  Section  3  above,  is 
currently  under  construction. 
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Figure  5:  Data  acquisition  dynamics  for  adaptive  imaging  spectrometer  as  demonstrated  by  spectral  imaging  objects  at  different 
temperatures  within  the  field  of  view  of  the  instrument. 
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Figure  6:  The  spectra  of  three  objects  in  the  field  of  view  of  the  imaging  spectrometer,  together  with  blackbody  curves  fitted  to  the 
temperatures  indicated. 


5.  CONCLUSIONS 

A  multispectral  imager  designed  to  simultaneously  produce  broadband  two-dimensional  imagery  for  target  tracking,  and 
multispectral  images  for  target  discrimination  is  described.  Under  software  control,  the  imager  can  perform  a  variety  of 
spectral  transforms  with  real-time  variable  data  acquisition  parameters.  Fast  adaptive  spectral  imagers  based  on  this 
approach  can  greatly  improve  the  discrimination  function  of  space  based  optical  sensors  by  allowing  real-time 
optimization  between  the  information  content  and  the  data  rate.  The  instrument’s  hardware  architecture  is  favorable  for 
space  applications,  since  it  contains  no  macro-scale  moving  parts  and  can  be  built  in  a  robust,  small-volume  and  low- 
mass  package.  The  technology  is  also  applicable  to  multispectral/hyperspectral  imaging  applications  in  diverse  areas 
such  as  surveillance,  atmospheric  remote  sensing,  process  control  and  biomedical  imaging,  and  can  be  adapted  for  use  in 
any  spectral  domain  from  the  ultraviolet  (UV)  to  the  LWIR  region. 
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